IES RV Institute of 
Amy Education Sciences 


AY Vesna Gresaleve-haceyel 


WHAT WORKS 
CLEARINGHOUSE™ 
yal AIAN) 


NCEE 2020-010 
U.S. DEPARTMENT OF EDUCATION 


A Publication of the National Center for Education Evaluation at IES 


Improving vocabulary and reading comprehension is critical 

to students’ academic success, particularly for English learners 
who may have less exposure to academic language than native 
English-speaking students.! Word Generation is a supplemental 
program that aims to improve students’ reading comprehen- 
sion by building students’ vocabulary, academic language, and 
perspective-taking skills through classroom discussion and 
debate. Word Generation was designed for all students; how- 
ever, English learners in particular could benefit from its focus 
on academic language. Word Generation consists of a series of 
interdisciplinary units with daily lessons focused on a high-in- 
terest topic to increase student engagement. Each unit targets a 
small number of academic vocabulary words that are integrated 
into texts, activities, writing tasks, debates, and discussions 
across content areas. Several Word Generation programs exist. 
In the Word Generation Weekly (WordGen Weekly) and Word 
Generation Elementary (WordGen Elementary) programs, units 


are designed to be used across English language arts, math, 
science, and social studies in grades 6-8 and grades 4 and 5, 
respectively. In the Science Generation (SciGen) and Social 
Studies Generation (SoGen) programs, units can supplement or 
be used in place of regular science and social studies curriculum 
units in grades 6-8. The different Word Generation programs 
can be implemented separately or together. 


This What Works Clearinghouse (WWC) report, part of the 
WWC’s English Learners topic area, explores the effects of Word 
Generation on English language proficiency and reading com- 
prehension for English learners. The WWC identified nine stud- 
ies of Word Generation that focused on English learners. One of 
these studies meets WWC standards. The evidence presented in 
this report is from one study of the impact of Word Generation 
on English learners in a range of grades—4th through 7th—and 
in a variety of school settings, including elementary, middle, and 


K-8 schools in urban, suburban, and small town districts. 


What Happens When Students Participate in Word Generation?' 


The evidence indicates that implementing Word Generation: 
¢ May result in little or no change in English language 
proficiency. 


¢ May result in little or no change in reading 
comprehension. 


Findings on Word Generation from the one study that meets 
WWC standards are shown in Table |. The table reports an 
effectiveness rating, the improvement index, and the number 
of studies and students that contributed to the findings. The 
improvement index is a measure of the intervention’s effect 
on an outcome. It can be interpreted as the expected change 
in percentile rank for an average comparison group student if 
that student had received the intervention. 


Table 1. Summary of findings on Word Generation from the one study that meets WWC standards 


Study Findings 


Evidence meeting WWC standards (version 4.0) 


Improvement index 


Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 
English language proficiency No discernible effects +9 1 241 
Reading comprehension No discernible effects +4 1 241 


Note: The improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention. 
For example, an improvement index of +9 means that the expected percentile rank of the average comparison group student would increase by 9 points if the student received 
Word Generation. The improvement index values are generated based on one finding from the outcome analysis that meets WWC standards conducted within each domain, as 
reported by Kim et al. (2018). .A positive improvement index does not necessarily mean the estimated effect is statistically significant. The English language proficiency outcome 
reported in the study is the Core Academic Language Skills Instruments (CALS-I). The reading comprehension outcome reported in the study is the Global Integrated Scenario- 
based Assessment (GISA). The effects of Word Generation are not known for other outcomes within the English Learners topic area, including alphabetics, reading fluency, 
general literacy achievement, writing conventions, writing productivity, writing quality, mathematics achievement, science achievement, and social studies achievement. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Primary Mathematics topic area protocol (version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 


effectiveness ratings and the statements used in this report: 


Effectiveness Rating Rating interpretation 


Description of the evidence 


Positive (or negative) effects The intervention is likely to change an Strong evidence of a positive effect, with no 


outcome 


overriding contrary evidence 


Potentially positive (or negative) effects The intervention may change an outcome Evidence of a positive effect with no overriding 


contrary evidence 


No discernible effects The intervention may result in little to no No affirmative evidence of effects 


change in an outcome 


Mixed effects The intervention has inconsistent effects Evidence includes studies in at least two of 


onan outcome 


How is Word Generation Implemented? 


The following section provides details of how Word Gener- 
ation was implemented. This information can help educators 
identify the requirements for implementing Word Generation 
and determine whether implementing this intervention would 
be feasible in their district or school. Information on Word 
Generation presented in this section comes from the study 
that meets WWC standards (Kim et al., 2018) and from cor- 
respondence with the developer. 


¢ Goal: Word Generation aims to improve students’ reading 
comprehension by building students’ vocabulary, academic 
language, and perspective-taking skills through classroom 
discussion and debate. 


¢ Target population: Word Generation can be used for 
students in grades 4 to 8. WordGen Weekly, SciGen, and 
SoGen are used with students in grades 6-8. WordGen 
Elementary is used with students in grades 4 and 5. 


¢ Method of delivery: WordGen Weekly and WordGen 
Elementary are supplemental interdisciplinary units that 
are integrated into English language arts, science, social 
studies, and mathematics instruction. SciGen and SoGen 
units can be used in place of the regular curriculum or as 
a supplement. Each unit includes additional activities for 
other content areas to provide cross-disciplinary reinforce- 
ment of academic vocabulary. 


¢ Frequency and duration of service: WordGen Weekly 
consists of three series of 24 weekly units, with each series 
designed to be used over one academic year. Series and 
units within series can be used in any order. Each unit 
within the series includes five target vocabulary words and 


these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


Comparison group: In the study that contributes to this 
intervention report, students in the comparison group 
were taught using a wide range of instructional prac- 


tices and standard programs that sought to improve 
vocabulary. These programs were highly variable 
across schools in the comparison group. 


five connected activities that typically last from 15 to 20 
minutes and take place during regular English language 
arts, science, social studies, and math instruction. 


WordGen Elementary consists of an introductory unit (one 
week) and 12 additional units (two weeks each) for each 
grade level (4 and 5). Each two-week unit includes daily 
40- to 50-minute lessons. Schools can choose to implement 
some or all of the units across the school year. 


SciGen and SoGen each offer six discipline-specific units 
for each grade level (6-8). Each unit is one week in length 
and includes daily 40- to 50-minute lessons in science 

or social studies plus 15- to 20-minute supplementary 
activities that can be integrated into English language arts, 
mathematics and science (for SoGen) or social studies (for 
SciGen) instruction. Schools can choose to implement 
some or all of the units across the school year. 


Intervention components: Word Generation includes 
several key components and features, as described in 
Table 2. The four different Word Generation programs can 
be implemented separately or together. 


Table 2. Components of Word Generation 


Core component Description 


Word Generation units | Each Word Generation unit is organized around a high-interest topic that aims to engage students’ interest—for example, 
“Should the school day be lengthened?” or “Should the government impose a mandatory year of service after high school?” 
Students read or listen to multiple points of view on a topic at the beginning of the week, learn target vocabulary words 
relevant to that topic, prepare and debate the topic, and finally write an argumentative piece on the topic, all using the 
target vocabulary words. Instruction prioritizes student participation and teamwork to improve other important skills such as 


academic language and perspective-taking. 


Cross-discipline 
activities 


Each Word Generation unit includes complementary activities across disciplines (English language arts, math, science, and 
social studies) to provide students with multiple opportunities to reinforce vocabulary learning and to use academic language 


in a variety of academic contexts. 


What Does Word Generation Cost? 


This preliminary list of costs is not designed to be exhaus- 
tive; rather, it provides educators an overview of the major 
resources needed to implement Word Generation. The 
program costs described below are based on the information 
available as of April 2019. 


¢ Equipment and materials costs: The student materials 
and teacher resources for implementing Word Generation 
can be downloaded for free and then printed and distrib- 
uted for noncommercial purposes from the program’s web- 
site (wordgen.serpmedia.org). As an option, students can 
access digital materials for Word Generation on tablets. 
There is no cost to download these materials or to access 
videos and other resources for teachers on vocabulary 
development and classroom discourse. School districts can 
purchase printed copies from a print-on-demand company 
linked to the program’s website. 


Personnel costs: Word Generation can be adopted without 
professional learning support from the developer, Strategic 
Education Research Partnership (SERP). However, SERP 
does offer professional development upon request to improve 
implementation fidelity. Trainings range from one to three 
days in duration. Costs range from $3,000 to $12,000 
(excluding travel and materials). 


Facilities costs: The program is delivered in a classroom 
setting during regular class time. 


Costs paid by students or parents: None. 
In-kind supports: No information is available. 


Sources of funding: School districts or schools usually 
purchase materials and/or professional development for 
Word Generation. 


For More Information: 


About Word Generation 
Strategic Education Research Partnership, 1100 Connecticut Ave NW, Suite 1310, Washington, DC 20036 
Attn: Suzanne Donovan 


Email: info@serpinstitute.org. Web: httos://wordgen.serpmedia.org/. Phone: (202) 223-8555 
About the cost of the intervention 
Web: https://wordgen.serpmedia.org/ 


In What Context Was Word Generation Studied? 


The following section provides information on the setting of mation can help educators understand the context in which 
the one study of Word Generation that meets WWC standards, the study of Word Generation was conducted, and determine 
and a description of the participants in the research. This infor- | whether the program might be suitable for their setting. 


WHERE THE STUDY WAS CONDUCTED 


A 1 study, 241 students in 3 districts the northeastern United States. 
we; In urban, suburban, and small town districts 


English Learners: 100% 


caus «KK 1 2 7 OOOO 


Grades 4—7 


LEARN MORE 


A Read more about the Word Generation intervention and the studies that are summarized here in the Intervention Report. 
Liww Contact Strategic Education Research Partnership for additional information on implementing Word Generation. 
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